System and method for polarization imaging

ABSTRACT

A system for polarization imaging comprises an optical diffuser characterized by a point spread function (PSF), an image sensor, a spatially multiplexed polarization filter between the optical diffuser and the image sensor, and an image processor. The image processor receives signals from the image sensor and reconstructs, based on the PSF, a separate image for each polarization direction formed on the polarization filter.

RELATED APPLICATION(S)

This application claims the benefit of priority under 35 USC § 119(e) ofU.S. Provisional Patent Application No. 63/342,751 filed on May 17,2022, the contents of which are all incorporated by reference as iffully set forth herein in their entirety.

The project leading to this application has received funding from theEuropean Research Council (ERC) under the European Union's Horizon 2020Research and Innovation Program (Grant Agreement No. 757497).

FIELD AND BACKGROUND OF THE INVENTION

The present invention, in some embodiments thereof, relates to opticsand, more particularly, but not exclusively, to a system and a methodfor polarization imaging.

Polarization is a basic property of electromagnetic waves, and isdefined as the angle and phase between the components of the electricfield. The human eye is sensitive to the intensity and the wavelength ofan electromagnetic wave in the visible range, but is not sensitive toits polarization. Yet, the polarization encapsulates information, and istherefore used in the field of optics.

Known in the art are polarization cameras that use the polarization ofthe light in order to reconstruct an image. These cameras can be broadlycategorized into two groups. The polarization cameras in one of thesegroup employ polarization filtering in a spatial multiplexed manner,wherein the focal plane is spatially divided into several polarizationdirection [see, e.g., U.S. Pat. No. 9,293,491]. Such spatialmultiplexing is similar in its principles to color mosaiced sensors. Thepolarization cameras the other group employ polarization filtering in atime-multiplexed manner, wherein the image is reconstructed from asequence of images, each captured from light that polarized along adifferent direction [see, e.g., U.S. Pat. No. 8,654,246].

SUMMARY OF THE INVENTION

According to an aspect of some embodiments of the present inventionthere is provided a system for polarization imaging. The systemcomprises an optical diffuser characterized by a point spread function(PSF), an image sensor, a spatially multiplexed polarization filterbetween the optical diffuser and the image sensor, and an imageprocessor having a circuit configured to receive signals from the imagesensor and to reconstruct based on the PSF a separate image for eachpolarization direction formed on the polarization filter.

According to some embodiments of the invention the system wherein theimage sensor comprises a plurality of wavelength channels, and whereinthe circuit of the image processor is configured to reconstruct eachseparate image using all wavelength channels.

According to some embodiments of the invention the polarization filtercomprises a plurality of spatial periods, each corresponding to adifferent region over the image sensor, such that a cross-section ofportion of a light beam passing through a particular period of thepolarization filter covers a respective region over the image sensor.

According to some embodiments of the invention the circuit of the imageprocessor is configured for combining the separate images to a combinedimage.

According to some embodiments of the invention the optical diffusercomprises a random diffusion pattern.

According to some embodiments of the invention the circuit of the imageprocessor is configured to reconstruct the image based only on signalsfrom a region over the image sensor at which the PSF is shift-invariant.

According to some embodiments of the invention the circuit of the imageprocessor is configured to reconstruct the image based on signals from aregion over the image sensor at which the PSF is shift-invariant, and aregion over the image sensor at which the PSF is shift-variant.

According to some embodiments of the invention the polarization filteris configured to apply only linear polarization.

According to some embodiments of the invention the polarization filteris configured to apply circular and/or elliptic polarization.

According to some embodiments of the invention the diffuser, the imagesensor, and the polarization filter are arranged such that light arrivesto the diffuser directly from a scene, then arrives to the arrived tothe polarization filter directly from the diffuser, then arrives to theimage sensor directly from the polarization filter.

According to some embodiments of the invention the system is a lenslesspolarization imaging system.

According to an aspect of some embodiments of the present inventionthere is provided a method of polarization imaging. The method comprisesdiffusing a light beam arriving from a scene according to a point spreadfunction (PSF) to provide a diffused light beam, applying spatiallymultiplexed polarization filtering to the diffused light to provide afiltered diffused light beam, generating electrical signals responsivelyto the filtered diffused light beam, and processing the electricalsignals to reconstruct, based on the PSF, a separate image for eachpolarization direction of the multiplexed polarization filtering.

According to some embodiments of the invention the method comprisesgenerating electrical signals over a plurality of wavelength channels,and wherein the processing comprises reconstruct each separate imageusing all wavelength channels.

According to some embodiments of the invention the application ofspatially multiplexed polarization filtering comprises applyingplurality of spatial periods, each corresponding to a different regionover an image sensor, such that a cross-section of portion of a lightbeam passing through a particular period covers a respective region overthe image sensor.

According to some embodiments of the invention the method comprisescombining the separate images to a combined image.

According to some embodiments of the invention the method employs arandom diffusion pattern.

According to some embodiments of the invention the reconstruction isbased only on signals from a portion of a cross-section of the beam atwhich the PSF is shift-invariant.

According to some embodiments of the invention the reconstruction isbased on signals from a portion of a cross-section of the beam at whichthe PSF is shift-invariant, and a portion of a cross-section of the beamat which the PSF is shift-variant.

According to some embodiments of the invention the applying thespatially multiplexed polarization filtering comprises applying onlylinear polarization.

According to some embodiments of the invention the applying thespatially multiplexed polarization filtering comprises applying circularand/or elliptic polarization.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the invention pertains. Although methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of embodiments of the invention, exemplarymethods and/or materials are described below. In case of conflict, thepatent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and are notintended to be necessarily limiting.

Implementation of the method and/or system of embodiments of theinvention can involve performing or completing selected tasks manually,automatically, or a combination thereof. Moreover, according to actualinstrumentation and equipment of embodiments of the method and/or systemof the invention, several selected tasks could be implemented byhardware, by software or by firmware or by a combination thereof usingan operating system.

For example, hardware for performing selected tasks according toembodiments of the invention could be implemented as a chip or acircuit. As software, selected tasks according to embodiments of theinvention could be implemented as a plurality of software instructionsbeing executed by a computer using any suitable operating system. In anexemplary embodiment of the invention, one or more tasks according toexemplary embodiments of method and/or system as described herein areperformed by a data processor, such as a computing platform forexecuting a plurality of instructions. Optionally, the data processorincludes a volatile memory for storing instructions and/or data and/or anon-volatile storage, for example, a magnetic hard-disk and/or removablemedia, for storing instructions and/or data. Optionally, a networkconnection is provided as well. A display and/or a user input devicesuch as a keyboard or mouse are optionally provided as well.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

Some embodiments of the invention are herein described, by way ofexample only, with reference to the accompanying drawings. With specificreference now to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

In the drawings:

FIG. 1 is a schematic illustration of a system for polarization imaging,according to some embodiments of the present invention;

FIGS. 2A-C are schematic cross-sectional illustrations of the systemshown in FIG. 1 , along the lines A—A, B—B, and C—C, respectively;

FIGS. 3A-F are schematic illustrations showing an overview of a lenslesspolarization camera used in experiments performed according to someembodiments of the present invention;

FIG. 4 shows an example of a point spread function (PSF) of a diffuserused in experiments performed according to some embodiments of thepresent invention;

FIG. 5 shows an image captured through a polarization filter used inexperiments performed according to some embodiments of the presentinvention;

FIGS. 6A-H show results of experiments in which the lenslesspolarization camera was used to image a front-illuminates sceneaccording to some embodiments of the present invention to study thesystem; and

FIGS. 7A-H show results of experiments in which the lenslesspolarization camera was used to image a back-illuminates scene accordingto some embodiments of the present invention to study the system.

DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION

The present invention, in some embodiments thereof, relates to opticsand, more particularly, but not exclusively, to a system and a methodfor polarization imaging.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the Examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways.

Referring now to the drawings, FIG. 1 illustrates a system 10 forpolarization imaging, according to some embodiments of the presentinvention.

System 10 can be used for imaging any scene 12 containing one or moreobjects 14 that vary the polarization of light 16 interacting therewithand provides a light beam 36 that has a polarization that is differentfrom the polarization of light 16. The light 16 can illuminate scene 12from its front side or from its back side. Thus, the light beam 36 thatis received by system 10 can be reflected off scene 12 or transmittedthrough scene 12, depending on the application for which system 10 isemployed. Representative examples of applications which can benefit fromsystem 100 include, without limitation, material analysis applications,structural analysis applications, astronomy, medical applications,surveillance applications, robotics, and the like.

FIG. 1 illustrates a side view of system 10, parallel to an optical axis24 thereof. The image plane of system 10 is therefore perpendicular tothe plane of FIG. 1 . Cross-sectional illustrations along the lines A—A,B—B, and C—C are illustrated in FIGS. 2A, 2B, and 2C, respectively.System 10 comprises an optical diffuser 18, an image sensor 22 having aplurality of sensing elements 31, and a spatially multiplexedpolarization filter 20 between optical diffuser 18 and image sensor 22.

In some embodiments of the present invention diffuser 18, image sensor22, and polarization filter 20 are arranged such that light 36 arrivesto diffuser 18 directly from scene 12, then arrives to polarizationfilter 20 directly from diffuser 18, and then arrives to image sensor 22directly from polarization filter 20. In some embodiments of the presentinvention system 10 is a lensless polarization imaging system that isdevoid of any lens.

Optical diffuser 18 may comprise a transparent or translucent substrate,such as, but not limited to, glass or transparent polymer, carrying apattern 26 designed to redirect light passing therethrough. In variousexemplary embodiments of the invention the pattern 26 is designed toredirect the light by scattering but may alternatively or additionallybe designed to redirect the light by diffraction. The pattern can beetched in the substrate or be attached thereto. The features of pattern26 may have any suitable shape including, without limitation, pyramids,cones, hemispheres, and the like. The pattern may include features ofone or more type, and their size can be uniform or it may vary acrossthe area of the substrate. Typically, the pattern's features aremicrometric structures.

As used herein, a “micrometric structure” describes solid structurewhich has at least one cross-sectional dimension and, in someembodiments, at least two or at least three orthogonal cross-sectionaldimensions less than 1 mm, or less than 500 microns, or less than 200microns, or less than 150 microns, or less than 100 microns, or evenless than 70, or less than 50, or less than 20, or less than 10, or lessthan 5 microns.

In some embodiments of the present invention the pattern of diffuser 18is a random pattern, but this need not necessarily be the case since insome application it may be desired to introduce order into the pattern,e.g., over a grid. For example, the pattern's features can be arrangedin circularly, elliptically, spirally, or linearly. When the featuresarranged linearly they are preferably arranged linearly along at leasttwo non-parallel directions.

The scattering elements are designed to scatter and homogenize lightincident on the diffuser. The pattern 26 of diffuser 18 can becharacterize by the point spread function (PSF) which mathematicallydescribes the response of diffuser 18 to a point light source, providingthe pattern of the light that is produced when a point source of lightis transmitted through diffuser 18.

The spread of the PSF preferably includes at least 15% or at least 20%or at least 25% of the width or the length of image sensor 22. The widthof at least a portion of the features of pattern 26, more preferably ofeach of the features of pattern 26, is preferably less than theaggregate width of N sensing elements (e.g., pixels) of image sensor 22,wherein N is at most 50 or at most 40 or at most 30 or at most 20 or atmost 10.

Diffuser 18 can be manufactured using any method known in the art, suchas, but not limited to, photolithography, imprint lithography, or thelike.

Image sensor 22 receives a light beam and responsively generates animage signal. Sensor 22 is preferably a pixelated sensor (e.g., a MOSimager, a CMOS imager or a CCD) wherein each pixel serves as a sensingelement 31 of sensor 22. The sensing elements 31 are preferably arrangedgrid-wise as an array 30 (e.g., a rectangular array). Image sensor 22can be a color sensor or a grey scale sensor. When sensor 22 is a colorsensor, the image signal is typically, but not necessarily, resolvedinto at least three wavelength channels. When there are three or morewavelength channels, three wavelengths are optionally and preferably inthe visible range, for example, a red channel, a green channel and ablue channel. Also contemplated are embodiments in which at least one ofthe wavelength channels is in the infrared range, and at least one ofthe wavelength channels is in the visible range. Also contemplated areembodiments in which at least one of the wavelength channels is in theultraviolet range, and at least one of the wavelength channels is in thevisible range. Also contemplated are embodiments in which at least oneof the wavelength channels is in the infrared range, and at least one ofthe wavelength channels is in the ultraviolet range. Also contemplatedare embodiments in which at least one of the wavelength channels is inthe infrared range, at least one of the wavelength channels is in thevisible range, and at least one of the wavelength channels is in theultraviolet range. The present embodiments also contemplate aconfiguration in which the imager provides image signal resolved intofour or more wavelength channels. For example, one of the fourwavelength channels can be in the infrared range (e.g., near infraredrange) and/or ultraviolet range (e.g., UVA, UVB, UVC range)) and each ofthe remaining three wavelength channels can be in the visible range.

A “visible range”, as used herein, refers to a range of wavelengths fromabout 400 nm to about 700 nm.

An “infrared range”, as used herein, refers to a range of wavelengthsfrom about 700 nm to about 1 mm.

A “near infrared range”, as used herein, refers to a range ofwavelengths from about 700 nm to about 1400 nm.

An “ultraviolet range”, as used herein, refers to a range of wavelengthsfrom about 100 nm to about 400 nm.

A “UVA range”, as used herein, refers to a range of wavelengths fromabout 320 nm to about 400 nm.

A “UVB range”, as used herein, refers to a range of wavelengths fromabout 280 nm to about 320 nm.

A “UVC range”, as used herein, refers to a range of wavelengths fromabout 100 nm to about 280 nm.

A representative example of a set of wavelength channels suitable forthe present embodiments is a red channel, corresponding to red light(e.g., light having a spectrum having an apex at a wavelength of about580-680 nm), a green channel, corresponding to green light (spectrumhaving an apex at a wavelength of from about 500 to about 580 nm), and ablue channel, corresponding to blue light (spectrum having an apex at awavelength of from about 420 to about 500 nm). Such a set of channels isreferred to herein collectively as RGB channels.

Another representative example of a set of wavelength channels suitablefor the present embodiments is a red channel, a green channel and a bluechannel as detailed above, and also an infrared channel corresponding tonear infrared light (spectrum having an apex at a wavelength of fromabout 800 to about 900 nm).

Spatially multiplexed polarization filter 20 preferably comprises apolarization pattern 28 that applies a position-dependent polarizationto light passing therethrough. The applied polarization is“position-dependent” in the sense that at least two different pointsover the surface of filter 20 polarize the light along a differentpolarization direction. This is conveniently achieved by providing apolarization pattern 28 that is formed of a plurality of polarizationelements each applying a specific polarization to the light passingthrough the respective polarization element. Preferably, but notnecessarily, each of at least a portion of the polarization elementsapplies linear polarization.

When image sensor 22 comprises a plurality of wavelength channels, thepolarization pattern 28 of polarization filter 20 optionally andpreferably comprises a plurality of spatial periods 32, eachcompromising a plurality of segments 38 and corresponding to a differentregion over of image sensor 22, such that a cross-section of portion ofa light beam passing through a particular period 32 covers therespective region over sensor 22. Preferably, the respective regionincludes pixels or sub-pixels of all wavelength channels, in a mannerthat there is a plurality of polarization directions to each wavelengthchannel and a plurality of wavelength channels to each polarizationdirection. Specifically, suppose that the particular period 32 has afirst segment that polarizes light along a first direction, and a secondsegment that polarizes light along a second direction. Suppose furtherthat the respective region of image sensor 22 comprises a firstplurality of pixels for a red channel, a second plurality of pixels fora green channel, and a third plurality of pixels for a blue channel. Inthis case, for each of the first, second and third plurality of pixels,one portion of the plurality of pixels receives light polarized by thefirst segment of the period and another portion of the plurality ofpixels receives light polarized by the second segment of the period.Generally, for N_(p) segments of period 32 and N_(c) wavelengthchannels, there are N_(p)N_(w) such combinations of polarizationdirection and wavelength channel.

In the representative example shown in FIG. 2B each period includes foursegments. The Inventors found that four segments are sufficient toreconstruct an image because they can provide sufficient informationallowing to extract all four Stokes parameters. However, any number ofperiods per segment can be used. For example, filter 20 can includethree, four, five, six or more segments per period. Further, FIG. 2Bshows an embodiment in which the segments of the period are in the formof stripes, each applying polarization along a different polarizationdirection, but other shapes and arrangements of the segments are alsocontemplated. Additionally, while FIG. 2B shows two periods, it is to beunderstood that any number of periods can be employed. Typically, thenumber of periods is at least the number of periods along a one of thedirections that define the array 30 of sensor 22. For example, whensensor 22 includes N×N pixels, then pattern 28 has at least N periods.

System 10 also comprises an image processor 34 having a circuitconfigured to receive signals from image sensor 22 and to reconstruct,based on the PSF of optical diffuser 18, a separate image for eachpolarization direction formed on polarization filter 20. Typically,image processor 34 employs a method that receives the PSF, a modeldescribing the polarization multiplexing of 20, and the image receivedfrom sensor 22, and generates an output having a plurality of imageseach estimating of the image of scene 12, for one of the polarizationdirections formed on polarization filter 20.

The method employed by image processor 34 employs optionally andpreferably integrates the imaging features of the PSF and the structureof the polarization filter into a joint reconstruction of a plurality ofpolarization sub-images. This can be done in more than one way. In oneexample a model-based inversion is employed. To reduce noiseamplification, image processor 34 optionally and preferably employs apolarization image prior. This is optionally and preferably done by amachine learning procedure that is trained to receive a polarizationimage prior and to output a denoised image in which the noise of theinput image is at least partially removed.

As used herein the term “machine learning” refers to a procedureembodied as a computer program configured to induce patterns,regularities, or rules from previously collected data to develop anappropriate response to future data, or describe the data in somemeaningful way.

Representative examples of machine learning procedures suitable for thepresent embodiments, include, without limitation, clustering,association rule algorithms, feature evaluation algorithms, subsetselection algorithms, support vector machines, classification rules,cost-sensitive classifiers, vote algorithms, stacking algorithms,Bayesian networks, decision trees, neural networks (e.g.,fully-connected neural network, convolutional neural network),instance-based algorithms, linear modeling algorithms, k-nearestneighbors (KNN) analysis, ensemble learning algorithms, probabilisticmodels, graphical models, logistic regression methods (includingmultinomial logistic regression methods), gradient ascent methods,singular value decomposition methods and principle component analysis.

Preferably, the machine learning procedure comprises an artificialneural network.

Artificial neural networks are a class of algorithms based on a conceptof inter-connected “neurons.” In a typical neural network, neuronscontain data values, each of which affects the value of a connectedneuron according to connections with pre-defined strengths, and whetherthe sum of connections to each particular neuron meets a pre-definedthreshold. By determining proper connection strengths and thresholdvalues (a process also referred to as training), a neural network canachieve efficient recognition of images and characters. Oftentimes,these neurons are grouped into layers in order to make connectionsbetween groups more obvious and to each computation of values. Eachlayer of the network may have differing numbers of neurons, and thesemay or may not be related to particular qualities of the input data.

In one implementation, called a fully-connected neural network, each ofthe neurons in a particular layer is connected to and provides inputvalue to those in the next layer. These input values are then summed andthis sum compared to a bias, or threshold. If the value exceeds thethreshold for a particular neuron, that neuron then holds a positivevalue which can be used as input to neurons in the next layer ofneurons. This computation continues through the various layers of theneural network, until it reaches a final layer. At this point, theoutput of the neural network routine can be read from the values in thefinal layer. Unlike fully-connected neural networks, convolutionalneural networks operate by associating an array of values with eachneuron, rather than a single value. The transformation of a neuron valuefor the subsequent layer is generalized from multiplication toconvolution. In various exemplary embodiments of the invention themachine learning procedure is a convolutional neural network (CNN).

The machine learning procedure used according to some embodiments of thepresent invention is a trained machine learning procedure, whichprovides output that is related non-linearly to the images with which itis fed.

A machine learning procedure can be trained according to someembodiments of the present invention by feeding a machine learningtraining program with noisy polarization images and respective denoisedimages from which the noise of the noisy polarization images has beenremoved. Once the data are fed, the machine learning training programgenerates a trained machine learning procedure which can then be usedwithout the need to re-train it.

In some embodiments of the present invention image processor 34 recoversan image x from a noisy and degraded image y=Hx+e, where H is adegradation model that provides a degraded image Hx, and e is anadditive noise. Preferably, image processor 34 solves an optimizationproblem formulated using an objective function that comprises a priorterm and a fidelity term, which comprises the norm (typically anEuclidean norm) of the distance between the noisy image y and thedegraded image Hx. The prior term is optionally and preferably obtainedusing the aforementioned machine learning procedure.

In some embodiments of the present invention the circuit of imageprocessor 34 reconstructs the image(s) based only on signals from aregion over image sensor 22 at which the PSF is shift-invariant. Theadvantage of these embodiments is that they add simplicity to thereconstruction process, because they allow a linear shift-invariant(LSI) model assumption. Alternatively, the circuit of image processor 34reconstructs the image(s) based on signals from a region over imagesensor at which PSF is shift-invariant, and a region over image sensorat which PSF is shift-variant. The advantage of these embodiments isthat they allow utilizing a duality between wide/narrow diffusion in aboarder angular range.

In embodiments in which image sensor 22 comprises a plurality ofwavelength channels, the circuit of image processor 34 reconstructs eachseparate image using all the wavelength channels of image sensor 22comprises. In some embodiments of the present invention the circuit ofimage processor 34 combining the separate images to a combined image.

System 10 can be used in many applications. In some embodiments of thepresent invention system 10 is used for imaging astronomical scenes. Forexample, system 10 can be used to study magnetic fields, since suchfields affect light polarization. By providing polarization images ofastronomical objects, such as pulsars or radio galaxies, the presenceand strength of magnetic fields can be inferred. In another example,system 10 is used to study the properties of interstellar andintergalactic dust. Since dust particles in space scatter and polarizelight in a manner that depends on their size, shape, and composition,such a polarization image of these particles can provide informationabout their properties.

System 10 can be used to capture polarization images of cables. Overtime, cables can become damaged due to wear and tear, corrosion, orexternal forces, such as vibrations or impacts. Since such damagesaffect the polarization of light interacting with the cable,polarization imaging can be used to detect and analyze these defects, aswell as to study the mechanical and electrical properties of cables. Oneway to use polarization imaging to analyze cables is by using polarizedlight microscopy. This involves shining polarized light onto the cableand observing the polarization of the light that is transmitted throughthe cable. By analyzing the polarization of the transmitted light, it ispossible to detect defects, such as cracks, corrosion, or materialinhomogeneities, that can affect the cable's mechanical or electricalproperties. Another way to use polarization imaging to analyze cables isby using polarimetric sensing. This involves transmitting polarizedlight through the cable and measuring the polarization of the light thatis reflected or transmitted by the cable. By analyzing the polarizationof the reflected or transmitted light, changes in the cable's geometryor material properties, as well as to detect defects, such as breaks,kinks, or twists, can be detected.

System 10 can also be used in material analysis, where it is used todetect defects or impurities in materials. For example, in thesemiconductor industry, system 10 can be used to detect defects insilicon wafers, which can affect the performance of electronic devices.Polarization images of these wafers provide information regarding thepolarization of the light that is transmitted through the wafer,allowing to detect defects and other imperfections.

System 10 can also be used for imaging of biological tissues, forexample, for the study of the structure and composition of biologicaltissues, such as skin, muscles, and tendons. In particular, thepolarization imaging of the present embodiments can provide informationabout the orientation of collagen fibers. System 10 can be employed inan endoscope and provide polarization images of capillary blood vesseland microscopic mucosa patterns on the mucosa surface layer with thedifference in their color tone emphasized. Examples of endoscope-relatedapplications that use some polarization images can also include sensingthe micro-geometry or surface micro-structure of the organ walls andenhancing the contrast of underwater scattered image in a capsuleendoscope, besides such observation into the object's organ mucosa.

System 10 can also be used for non-destructive testing of materials,such as, but not limited to, composites, ceramics, and metals, in orderto facilitate detection of defects, stress, and strain. Polarizationimages captured by system 10 can be used to study the deformation andfailure mechanisms of materials under load.

Images captured by system 10 can also be used to detect hidden objectsby analyzing the polarization of the reflected light. It can also beused to enhance the contrast of images taken in low-light conditions.

System 10 can also be employed in the field of robotics, for example, toaid in navigation and object recognition, both in indoor and outdoorenvironments with varying lighting conditions. It can also be used tostudy the behavior of insects and other animals that use polarizationcues for navigation.

System 10 can also be used in the analysis of internal structures. Forexample, system 10 can be used in polarized light microscopy. In thistechnique, polarized light is used to illuminate the sample, and apolarizer is used to selectively filter the light that is transmittedthrough the sample. System 10 images the transmitted light analyzing,and the obtained image is analyzed to obtain information about theorientation, composition, and shape of the internal structures in thesample.

As used herein the term “about” refers to ±10%.

The terms “comprises”, “comprising”, “includes”, “including”, “having”and their conjugates mean “including but not limited to”.

The term “consisting of” means “including and limited to”.

The term “consisting essentially of” means that the composition, methodor structure may include additional ingredients, steps and/or parts, butonly if the additional ingredients, steps and/or parts do not materiallyalter the basic and novel characteristics of the claimed composition,method or structure.

As used herein, the singular form “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. For example,the term “a compound” or “at least one compound” may include a pluralityof compounds, including mixtures thereof.

Throughout this application, various embodiments of this invention maybe presented in a range format. It should be understood that thedescription in range format is merely for convenience and brevity andshould not be construed as an inflexible limitation on the scope of theinvention. Accordingly, the description of a range should be consideredto have specifically disclosed all the possible subranges as well asindividual numerical values within that range. For example, descriptionof a range such as from 1 to 6 should be considered to have specificallydisclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numberswithin that range, for example, 1, 2, 3, 4, 5, and 6. This appliesregardless of the breadth of the range.

Whenever a numerical range is indicated herein, it is meant to includeany cited numeral (fractional or integral) within the indicated range.The phrases “ranging/ranges between” a first indicate number and asecond indicate number and “ranging/ranges from” a first indicate number“to” a second indicate number are used herein interchangeably and aremeant to include the first and second indicated numbers and all thefractional and integral numerals therebetween.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

Various embodiments and aspects of the present invention as delineatedhereinabove and as claimed in the claims section below find experimentalsupport in the following examples.

EXAMPLES

Reference is now made to the following examples, which together with theabove descriptions illustrate some embodiments of the invention in a nonlimiting fashion.

Polarization imaging is a technique that creates a pixel-map of thepolarization state in a scene. Although invisible to the human eye (andfor most conventional cameras), polarization can assist various sensingand computer vision tasks. Existing polarization cameras acquire theadditional modality using either spatial or temporal multiplexing, whichincreases the camera volume, weight, cost, or altogether. Recentlensless imaging techniques presented a way of image acquisition withouta lens in a potentially miniature scale element. By utilizing an opticalelement that performs a smart coding of the scene on the sensor, a sharpimage reconstruction is achieved. This example, describes a lenslesspolarization camera, composed of a diffuser and a polarizationmultiplexing filter. By combining it with a reconstruction algorithmthat accounts for the compressed imaging properties, good polarizationimages were obtained.

INTRODUCTION

Polarization is a basic property of light (and electro-magnetic waves ingeneral), defined as the angle and phase between the components of theelectric field. It encapsulates beneficial information for varioussensing and machine vision applications, as mapping the polarizationstate of a scene can enhance the imaging conditions or measure thingsthat are conventionally invisible. For example, it can be utilized tomap and measure mechanical strain in transparent materials, which isinvisible in conventional cameras.

Existing polarization cameras are based either on division of focalplane (similar to color mosaiced sensors), or sequential polarizationfiltering. Polarization-mosaiced sensors [1] enable nice reconstructionresults, by utilizing the tremendous effort invested in demosaicingalgorithms [2, 3] (originally developed for color imaging). Yet, theInventors found that they require complex chip-level fabrication. Thesequential filtering acquisition approach is based on a conventionalimage sensor with a polarizer replacement mechanism designed toreplace/rotate a polarizer, so the required polarization sub-images canbe acquired sequentially. The Inventors found that this scheme limitsthe allowed dynamics in the scene and/or camera shake (due toregistration issues). To avoid this limitation, a rapid polarizermechanism can be used [4], but the Inventors found that its volume,weight and cost are relatively high.

The Inventors recognized that extreme imaging application like compactmicroscopy and endoscopy, which may benefit from polarization imaging[5-7], have very strict space and weight requirements, preventing theuse of complicated optics, or even the use of a lens [8]. To addressthis challenge, the Inventors devised a lensless imaging solution.

In the lensless imaging scheme, a hybrid optical-digital acquisition isperformed, where an optical element generates a coded image of the sceneon the sensor, and a post-processing algorithm reconstructs the sharpimage. Such lensless cameras can even acquire additional modality alongwith the 2D images, such as depth [9-11], time [12] or spectrum [13].All of these lensless solutions significantly reduce the system sizewhile allowing the acquisition of these modalities

FIGS. 3A-F illustrate an overview of the lensless polarization camera ofthe present example. Assuming a (A) scene with polarized light, theimaging is performed using (B) a random diffuser, and polarizationmultiplexing is achieved using (C) a polarization multiplexing filterlocated in the sensor plane. This imaging scheme results in (D) adiffused and multiplexed intermediate image. The diffuser generates aunique PSF with widely spread narrow features (see FIG. 4 ), allowingimage reconstruction from a partially sampled image; this feature allowsa simple polarization filter design. Utilizing the diffuser PSF and thepolarization filter characteristics (E), the reconstruction algorithmcan restore the (F) polarization image of the scene.

The camera of the present example includes (i) a random diffuser(similar to the one used in [9]), whose Point Spread Function (PSF)contains widely spread narrow features, enabling image reconstructionfrom a partially sampled image; (ii) a polarization filter designed formultiplexing the required polarization states; and (iii) an imageprocessor employing a reconstruction algorithm tailored to the uniquefeatures of polarization images.

The camera of the present example has the following properties: (i)compactness; (ii) low-cost; (iii) requiring only a single shot; and (iv)providing good polarization images. The last point is demonstratedherein both in simulation and using a prototype in experiments.Specifically, it is demonstrated that it can be useful for real worldapplications, specifically, for strain analysis.

Recently, several new polarization cameras were introduced [1, 4, 17],which led to a renewed interest in polarization imaging both in researchand applications. The unique properties uncovered using polarizationimaging lead to many applications, for example in extreme imagingconditions [18, 19], biological/medical applications [5-7], computervision tasks [20, 21], remote sensing [22] and even exoplanets discoveryin astronomy [23]. This proliferation also led to research on algorithmsfor polarization image processing, for example demosaicing of apolar-mosaiced sensor (either monochrome or color) [2, 3, 24], anddedicated denoising [25, 26], and deblurring [27] methods forpolarization images.

Polarization Imaging

Polarization imaging is defined as the pixelwise mapping of thepolarization state in a scene. It can be fully defined using the fourStokes parameters [14]:

$\begin{matrix}\left\{ {\begin{matrix}{S_{0} = I} \\{S_{1} = {{Ip}\cos\left( {2\psi} \right)\cos\left( {2\xi} \right)}} \\{S_{2} = {{Ip}\sin\left( {2\psi} \right)\cos\left( {2\xi} \right)}} \\{S_{3} = {{Ip}\sin\left( {2\xi} \right)}}\end{matrix},} \right. & \left( {{EQ}.1} \right)\end{matrix}$

where S_(i) is the ith Stokes parameter, 1 is the total intensity, p isthe degree of polarization and ψ, ξ are the angles of the polarizationpoint on the Poincaré sphere [15], indicating the angle and ellipticityof the polarization state. These parameters can be calculated for everypoint in the scene using several intensity images taken in differentpolarization states [16] (which will be coined hereafter as polarizationsub-images):

$\begin{matrix}\left\{ {\begin{matrix}{S_{0} = {I_{0} + I_{90}}} \\{S_{1} = {I_{0} - I_{90}}} \\{S_{2} = {I_{45} - I_{I35}}} \\{S_{3} = {I_{RCP} - I_{LCP}}}\end{matrix},} \right. & \left( {{EQ}.2} \right)\end{matrix}$

where I_(θ) is the sub-image in linear polarization at angle θ° or inright/left circular polarization (RCP/LCP).

Lensless Imaging

Several lensless imaging methods were recently introduced [9-11, 28-31].Such cameras are based on a conventional image sensor and a mask thatreplaces the lens in the sense that it has some sort of a PSF. Such PSFis generally very poor and its imaging capability is far from anyconventional lens performance. However, such mask has unique propertiesthat enable a post-capture reconstruction of the image, achieving nicelenssless imaging performance. The mask can modulate either amplitude[10] or phase [9, 11, 31], be designed [10, 11] or random [9], and insome cases even a bare sensor [30] or fiber bundle [28, 29] can suffice.In addition to conventional 2D imaging, such lensless cameras cancapture additional modalities, e.g. depth [9-11], time [12] or spectrum[13]. In this example, the unique features of lensless imaging areefficiently utilized together with the properties of polarization imagesto design a simple lensless polarization camera.

Plug-and-Play Image Restoration

Non-blind image reconstruction is based on model-based inversion andprior compliance. While the first is generally theoretically reasonedand rigorous, the Inventors found that it suffers from several inherentissues, where the most prominent are noise sensitivity and modelinaccuracies. These limitations result in poor reconstructionperformance in almost all real-world applications. To address theseissues, model-based inversion is balanced with prior compliance, whichis designed to regularize the data-term, and generate an output imagewith similar appearance of the image domain (which can be monochrome,color, infrared, multi/hyper-spectral, polarization etc.). While themodel-prior balance generates an inherent trade-off [32], the prior termitself, and its integration in the reconstruction process is an activearea of research for decades [33-36].

Recently, end-to-end trained deep-learning (DL) based methods weresuggested as a solution for inverse problems [37]. Such methodencapsulate both the model and the prior, without any clear separationbetween them. This approach generally leads to improved results, butdepends on very large training sets, and require dedicated training ofthe DL-model for every degradation model. These issues can be addressedusing un/self-supervised learning [38, 39].

Another approach, coined Plug-and-Play (PnP) priors for imagerestoration [40], which has many recently proposed variants [41-44],attempts to take the best of both worlds, i.e. the accuracy, flexibilityand explainability of model-based reconstruction, along with astandalone (preferably learnable) denoiser prior. These methods use adenoising operation as a prior term. Intuitively, the denoisingoperation can be considered also as a projection operator to themanifold of desired images. By iteratively performing model-basedinversion and then denoising, improved reconstruction results can beachieved. The method features a very attractive flexibility vs.performance trade-off, as both the denoiser (either learnable or not)and the degradation model can be easily replaced, and simple parametertuning can balance the data/prior trade-off.

System Design

The lensless polarization camera design of this example is based on acombination of a random diffuser with a polarization multiplexingfilter. These two components encode the polarization information in theimage plane. The diffuser replaces the lens thanks to its unique PSFthat performs a spatial encoding of the scene, which enables itsreconstruction even from a partially sampled image. This unique spatialcoding allows using a simple polarization filter to multiplex thedifferent polarization states, thus allowing to restore the polarizationsub-images which are required to perform the polarization mapping of thescene. To reconstruct the polarization sub-images, a dedicated PnP-basedalgorithm that relies on the resulted encoding and the statistics ofpolarization images is employed.

Diffuser Based Imaging

FIG. 4 shows an example diffuser PSF. The random pattern of the diffuseracts as multiple randomly distributed lens-like features. This providesa widely spread PSF with many narrow features, which enables imagereconstruction.

Due to its random structure, the diffuser's PSF has both very widespatial response, containing a lot of narrow caustic-like features (seeFIGS. 2A-C). Each narrow feature in the PSF can be considered as theresponse of a micro-lens (as it maps a point-source to a point in theimage), and therefore the full intermediate image formed by the diffusercan be thought of as numerous shifted replicas of the object, spread onalmost the entire sensor due to the wide extent of the PSF.

This uncommon wide/narrow duality of the diffuser PSF enables a uniquelensless imaging scheme: as the intermediate image contains multipleshifted replicas of the scene, even partial sampling of the image planecan contain enough information for image reconstruction. This principleis utilized for the compression and restoration of an additionalmodality along with the 2D images. A time/spectrum reconstruction [12,13] is used for polarization reconstruction. To simplify thereconstruction process, the imaging system was limited to the domainwhere the PSF is shift-invariant, thus enabling a linear shift-invariant(LSI) model assumption. However, One may use a shift-variant PSF of thediffuser, as it features the wide/narrow duality in a very wide angle.Thus, the solution in this example can be adapted also with a linearshift-variant model.

Polarization Multiplexing Filter

The unique wide/narrow duality of the diffuser PSF enables imagereconstruction even from a partially sampled image. Therefore, spatialmultiplexing in the image plane can be used to compress an additionalmodality along with the spatial information. This compression abilityhad been presented for video from single image using the spatial rollingshutter sampling pattern [12] and for multi-spectral reconstructionusing a mosaiced spectral filter [13]. In this work, the same principleis used for multiplexing polarization sub-images.

While polarization is a property of the electro-magnetic wave amplitude,conventional image sensors sense only the intensity, and they areinsensitive to the polarization. Therefore, to enable polarizationimaging, polarization optical elements are used so as to generatepolarization sensitivity, and several intensity images are captured soas to solve the amplitude-to-intensity ambiguities. Full reconstructionof the Stokes parameters for each pixel requires six polarizationsub-images, as presented in EQ. (2). Yet, linear-only polarizationmapping can be achieved using only four sub-images. In this example, thereconstruction is only for the case of linear polarization mapping, butsimilar approach can be used for circular or elliptical polarization.

For linear polarization mapping, four polarization sub-images arerequired: I_(θ), where θ=0°, 45°, 90°, 135° (note that while linearpolarization mapping is also possible using three sub-images, thisexample follows the common practice and uses four, to improve stability[1, 4, 17]).

To avoid complex pixel-level fabrication, and by leveraging the partialsampling ability of the diffuser-based imaging scheme, a very simplestriped structure polarization multiplexing filter is used, with aperiodic pattern of linear polarizer stripes in the required angles.FIG. 5 is an image captured through a filter composed of severalpolarizer stripes in the required angles. Captured under polarizedillumination, the different transmission can be clearly seen.

The structure of the filter allows partial sampling in each of therequired polarization angles. The main trade-off in such a filter designis the individual stripe width, where a smaller stripe performs a densersampling, with the cost of a more complex fabrication. As very densesampling is not critical, relatively wide stripes, which are easy tofabricate, can be used, as detailed below.

Reconstruction Algorithm

In this example, the reconstruction algorithm integrates the diffuserPSF imaging features and the polarization multiplexing filter structureto a joint reconstruction of the four polarization sub-images. Whilemodel-based inversion (utilizing the known LSI diffuser PSF and thepolarization filter structure) can lead to some reconstruction ability,such an approach suffer from very high noise sensitivity, resulting innoise amplification and performance degradation. To overcome this, somesignal prior is be used.

As shown in FIGS. 6A-H and 7A-H below, polarization images have uniqueproperties, which are not easily derived from natural image priors. Someedges are consistent to all sub-images, while others depend on thematerials and lighting properties. Moreover, the material and lightingrelated edges/textures have inter sub-image correlations, related to thepolarization properties of the material.

In order to implicitly grasp the complex structure of polarizationimages and use it as a regularizing prior, the concept of PnP isemployed. A deep CNN for polarization image denoising is trained, andits denoising operation is integrated in the reconstruction process as apolarization image prior.

The method is based on the framework for recovering a signal x from adegraded and noisy measurement y=Hx+e, where H is the degradation modeland e is the additive noise:

$\begin{matrix}{{f\left( \overset{\sim}{x} \right)} = {{\frac{1}{2\sigma_{e}^{2}}{{y - {H\overset{\sim}{x}}}}_{2}^{2}} + {{s\left( \overset{\sim}{x} \right)}.}}} & \left( {{EQ}.3} \right)\end{matrix}$

The problem is formulated as a cost function f(x), where {tilde over(x)} is the optimization variable, and the symbol ∥·∥₂ stands for theEuclidean norm. The first term in EQ. 3 is the fidelity term thatvalidates model consistency. The second is the prior term thatregularizes the optimization process using the image model s(x). Theminimization of the cost function converges to the desired solution. Assolving f(x) directly is not simple, it can be split to several termsand alternately optimized using either the ADMM [49] or FISTA [50]methods. The PnP method [40] suggest to replace the prior-related stepwith a gaussian denoiser. Using this approach, the degradation model andthe denoiser prior can be easily replaced, allowing high flexibility andexplainability. The full process, based on FISTA steps with apolarization image denoising CNN, is presented in Procedure 1, below.

Procedure 1 Input: Diffuser PSF and polarization filter multiplexingmodel H; diffused and multiplexed image y, polarization image denoisingCNN 

(·; σ); stopping criterion. Output: {circumflex over (x)} an estimatefor x. Initialize: x₀, v₀ = some initialization; t₀ = 1, k = 0; someinitialization for σ_(k) and λ. while stopping criterion not met do | k= k + 1; |${x_{k} = {v_{k - 1} - {\frac{1}{\lambda}{H^{T}\left( {{Hv}_{k - 1} - y} \right)}}}};$| x_(k) = 

(x_(k) ; σ_(k)); | t_(k) = 1 + {square root over (1 + 4t_(k−1) ²)}/2; |${v_{k} = {x_{k} + {\frac{t_{k - 1} - 1}{t_{k}}\left( {x_{k} - x_{k - 1}} \right)}}};$end {circumflex over (x)} = x_(k);

The initialization of x₀, v₀ can be set to zero. However, for improvedconvergence, one can initialize with simple restoration based oninterpolation (to fill the polarization filter gaps) and Wienerfiltering. The noise standard-deviation σ_(k) can be set to convergefrom an initial value, or be estimated in every iteration using theresidual error:

e _(k) =|Hv _(k-1) −y| ₂.

The stopping criterion that was tested is e_(k)<e_(desired) orN_(iter)<N_(max), where e_(desired) was set according to the tolerablenoise level in the current scene, and N_(max) was set following themaximal time the procedure was allowed to converge.

The polarization image denoising CNN, which is used as a prior in thereconstruction process, is based on the denoiser architecture presentedin [44] that achieve state of the art performance for color imagedenoising. The architecture is adapted to polarization images andtrained using the polarization image datasets from KAUST [3] andTokyoTech [24]. A total of 4480 patches of 128×128 pixels are used, in a80/20 training/validation split. As the architecture was designed to getthe noise standard deviation as an input, the image patches were noisedduring training with additive white Gaussian noise with σ˜U(1,50) (on a[0-255] scale). The CNN is trained using the smooth L1 loss and ADAMoptimizer for 300 epochs. The CNN achieves nearly perfect denoisingperformance of δ<5, and gradually degrades to PSNR=33, SSIM=0.93 forσ=50.

Experimental Results

To experimentally validate the proposed design, a lensless polarizationcamera prototype was built. Its components and assembly process aredetailed, and its performance on real world polarized scenes isanalyzed. Following the performance analysis, a discussion on theapproach limitations is brought.

Prototype Structure

The prototype lensless polarization camera was based on a 0.5° diffuser(Edmund Optics #35-860) mounted on a 12.3MP, 3.45 μm pixel pitch CMOScamera (Thorlabs CS126CU). The polarization filter is fabricated using alinear film polarizer (Thorlabs LPVISE2X2) cut to stripes ofapproximately 880 μm (equivalent to about 256 pix) oriented in therequired polarization angles. The stripes were then assembled in therequired form, and the filter is incorporated in front of the sensor.The diffuser PSF was measured by taking an image of a point source (FIG.4 ) without the polarization filter. The polarization filter responsewas measured using the bare image sensor and a polarized light sourcerotated in the required polarization angles. The measured PSF andpolarization filter response were incorporated in the reconstructionprocedure as the degradation and polarization multiplexing models.

Table-Top Experiments

To demonstrate and analyze the capabilities of the proposed camera, twotypes of polarized light scenes were created: (i) front-illuminatedscene with two orthogonally polarized projectors, to demonstrate theability to separate between two sources of light, and (ii)back-illuminated scene with a polarized screen, with a transparentmaterial on it, to analyze the ability to perform strain analysis. Inboth cases, the performance are compared to a conventional lens camera,performing polarization imaging using sequential acquisition.

In the case of front-illuminates scene (FIGS. 6A-H), the scene wasilluminated with a linear polarized light at θ=0° from the left, and asimilar light at θ=90° from the right. In such case, I₀, I₉₀ look likethey are lit from opposite directions, while 145, 1135 are evenlyilluminated. FIGS. 6A-H show a front illumination example. FIG. 6A-Dshow lensless imaging results and FIGS. 6E-H show reference lens-basedcamera for different polarization sub-images: I₁₃₅ (FIGS. 6A and 6E),190 (FIGS. 6B and 6F), 145 (FIGS. 6C and 6G), I₀ (FIGS. 6D and 6H).

As shown in FIGS. 6A-H, the overall structure of the scene is wellreconstructed, with typical artifacts to lensless images. The evenillumination for I₄₅, I₁₃₅ is well reconstructed, and the unevenillumination in I₀, I₉₀ is also visible although not as prominent as inthe reference lens-based images.

Another common application of polarization imaging is strain analysis intransparent materials. Non-contact strain analysis is a challenge, whichcan be efficiently addressed with polarization imaging due to the factthat strain causes birefringence, which changes the polarization stateof the light. Since birefringence is also highly wavelength dependent(in most cases), color-polarization imaging can be used for mapping it,and as a byproduct the strain can be analyzed. Back illumination exampleis presented in FIGS. 7A-H, where a plastic bag is located on a whitelinear polarized screen (θ=135°). FIG. 7A-D shows lensless imagingresults and FIGS. 7E-H shows reference lens-based camera for differentpolarization sub-images: I₁₃₅ (FIGS. 7A and 7E), 190 (FIGS. 7B and 7F),145 (FIGS. 7C and 7G), I₀ (FIGS. 7D and 7H). As shown, most of theeffect takes place in 145, 1135, and its overall structure and color(indicating the strain direction and level) are well reconstructed. InI₀, I₉₀ the birefringence is less prominent, and the overall structureis reconstructed with lensless imaging artifacts. Note that thereference background illumination point (top-right in each sub-image) isreconstructed according to the polarization direction.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims.

It is the intent of the applicant(s) that all publications, patents andpatent applications referred to in this specification are to beincorporated in their entirety by reference into the specification, asif each individual publication, patent or patent application wasspecifically and individually noted when referenced that it is to beincorporated herein by reference. In addition, citation oridentification of any reference in this application shall not beconstrued as an admission that such reference is available as prior artto the present invention. To the extent that section headings are used,they should not be construed as necessarily limiting. In addition, anypriority document(s) of this application is/are hereby incorporatedherein by reference in its/their entirety.

REFERENCES

-   [1] Sony, “Polarsens,”    https://www(dot)sony-semicon(dot)co(dot)jp/e/products/IS/industry/technology/polarization(dot)html    (2021). Accessed: 2021-12-24.-   [2] J. Zhang, J. Shao, H. Luo, X. Zhang, B. Hui, Z. Chang, and R.    Liang, “Learning a convolutional demosaicing network for microgrid    polarimeter imagery,” Opt. Lett. 43, 4534-4537 (2018).-   [3] S. Qiu, Q. Fu, C. Wang, and W. Heidrich, “Polarization    Demosaicking for Monochrome and Color Polarization Focal Plane    Arrays,” in Vision, Modeling and Visualization, H.-J. Schulz, M.    Teschner, and M. Wimmer, eds. (The Eurographics Association, 2019).-   [4] N. Lefaudeux, N. Lechocinski, S. Breugnot, and P. Clemenceau,    “Compact and robust linear Stokes polarization camera,” in    Polarization: Measurement, Analysis, and Remote Sensing VIII, vol.    6972 D. B. Chenault and D. H. Goldstein, eds., International Society    for Optics and Photonics (SPIE, 2008), pp. 76-87.-   [5] M. Shribak, “Polychromatic polarization microscope: bringing    colors to a colorless world,” Sci. reports 5, 17340 (2015).-   [6] J. C. Ramella-Roman, I. Saytashev, and M. Piccini, “A review of    polarization-based imaging technologies for clinical and preclinical    applications,” J. Opt. 22, 123001 (2020).-   [7] C. Rodríguez, A. V. Eeckhout, L. Ferrer, E. Garcia-Caurel, E.    González-Arnay, J. Campos, and A. Lizana, “Polarimetric data-based    model for tissue recognition,” Biomed. Opt. Express 12, 4852-4872    (2021).-   [8] K. Yanny, N. Antipa, W. Liberti, S. Dehaeck, K. Monakhova, F. L.    Liu, K. Shen, R. Ng, and L. Waller, “Miniscope3d: optimized    single-shot miniature 3d fluorescence microscopy,” Light. Sci. &    Appl. 9 (2020).-   [9] N. Antipa, G. Kuo, R. Heckel, B. Mildenhall, E. Bostan, R. Ng,    and L. Waller, “Diffusercam: lensless single-exposure 3d imaging,”    Optica 5, 1-9 (2018).-   [10] M. S. Asif, A. Ayremlou, A. Sankaranarayanan, A. Veeraraghavan,    and R. G. Baraniuk, “Flatcam: Thin, lensless cameras using coded    aperture and computation,” IEEE Transactions on Comput. Imaging 3,    384-397 (2017).-   [11] V. Boominathan, J. Adams, J. Robinson, and A. Veeraraghavan,    “Phlatcam: Designed phase-mask based thin lensless camera,” IEEE    Transactions on Pattern Analysis Mach. Intell. pp. 1-1 (2020).-   [12] N. Antipa, P. Oare, E. Bostan, R. Ng, and L. Waller, “Video    from stills: Lensless imaging with rolling shutter,” in 2019 EEE    International Conference on Computational Photography (ICCP),    (2019), pp. 1-8.-   [13] K. Monakhova, K. Yanny, N. Aggarwal, and L. Waller, “Spectral    diffusercam: lensless snapshot hyperspectral imaging with a spectral    filter array,” Optica 7, 1298-1307 (2020).-   [14] F. Perrin, “Polarization of light scattered by isotropic    opalescent media,” The J. Chem. Phys. 10,415-427 (1942).-   [15] H. Poincaré, “Théorie mathématique de la lumière, vol. 2    (georges carré, paris),” MI MISHCHENKO AND LD TRAVIS 44 (1892).-   [16] J. E. Solomon, “Polarization imaging,” Appl. Opt. 20,1537-1544    (1981).-   [17] N. A. Rubin, G. D′Aversa, P. Chevalier, Z. Shi, W. T. Chen,    and F. Capasso, “Matrix fourier optics enables a compact full-stokes    polarization camera,” Science 365, eaax1839 (2019).-   [18] Y. Zhu, T. Zeng, K. Liu, Z. Ren, and E. Y. Lam, “Full scene    underwater imaging with polarization and an untrained network,” Opt.    Express 29, 41865-41881 (2021).-   [19] J. Fade, S. Panigrahi, A. Carré, L. Frein, C. Hamel, F.    Bretenaker, H. Ramachandran, and M. Alouini, “Long-range    polarimetric imaging through fog,” Appl. Opt. 53,3854-3865 (2014).-   [20] V. Deschaintre, Y. Lin, and A. Ghosh, “Deep polarization    imaging for 3d shape and svbrdf acquisition,” in Proceedings of the    IEEE/CVF Conference on Computer Vision and Pattern Recognition    (CVPR), (2021).-   [21] Y. Ba, A. Gilbert, F. Wang, J. Yang, R. Chen, Y. Wang, L.    Yan, B. Shi, and A. Kadambi, “Deep shape from polarization,” in    Computer Vision—ECCV 2020, A. Vedaldi, H. Bischof, T. Brox, and    J.-M. Frahm, eds. (Springer International Publishing, Cham, 2020),    pp. 554-571.-   [22] J. S. Tyo, D. H. Goldstein, D. B. Chenault, and J. A. Shaw,    “Polarization in remote sensing—introduction,” Appl. Opt.    45,5451-5452 (2006).-   [23] H. M. Schmid, D. Gisler, F. Joos, H. P. Povel, J. O.    Stenflo, M. Feldt, R. Lenzen, W. Brandner, J. Tinbergen, A.    Quirrenbach, R. Stuik, R. Gratton, M. Turatto, and R. Neuhäuser,    “ZIMPOL/CHEOPS: a Polarimetric Imager for the Direct Detection of    Extra-solar Planets,” in Astronomical Polarimetry: Current Status    and Future Directions, vol. 343 of Astronomical Society of the    Pacific Conference Series A. Adamson, C. Aspin, C. Davis, and T.    Fujiyoshi, eds. (2005), p. 89.-   [24] M. Morimatsu, Y. Monno, M. Tanaka, and M. Okutomi, “Monochrome    and color polarization demosaicking using edge-aware residual    interpolation,” 2020 IEEE Int. Conf. on Image Process. (ICIP) pp.    2571-2575 (2020).-   [25] J. Zhang, H. Luo, R. Liang, W. Zhou, B. Hui, and Z. Chang,    “Pca-based denoising method for division of focal plane    polarimeters,” Opt. Express 25, 2391-2400 (2017).-   [26] X. Li, H. Li, Y. Lin, J. Guo, J. Yang, H. Yue, K. Li, C. Li, Z.    Cheng, H. Hu, and T. Liu, “Learning-based denoising for polarimetric    images,” Opt. Express 28, 16309-16321 (2020).-   [27] A. Tang, “A restoration of underwater polarized images based on    dcp,” in 2019 International Conference on Sensing, Diagnostics,    Prognostics, and Control (SDPC), (2019), pp. 675-678.-   [28] A. Porat, E. R. Andresen, H. Rigneault, D. Oron, S. Gigan,    and O. Katz, “Widefield lensless imaging through a fiber bundle via    speckle correlations,” Opt. Express 24, 16835-16855 (2016).-   [29] U. Weiss and O. Katz, “Two-photon lensless micro-endoscopy with    in-situ wavefront correction,” Opt. Express 26, 28808-28817 (2018).-   [30] G. Kim, K. Isaacson, R. Palmer, and R. Menon, “Lensless    photography with only an image sensor,” Appl. Opt. 56, 6450-6456    (2017).-   [31] A. Sinha, J. Lee, S. Li, and G. Barbastathis, “Lensless    computational imaging through deep learning,” Optica 4, 1117-1125    (2017).-   [32] Y. Blau and T. Michaeli, “The perception-distortion tradeoff,”    in The IEEE Conference on Computer Vision and Pattern Recognition    (CVPR), (2018).-   [33] A. N. Tikhonov and V. Y. Arsenin, Solutions of ill-posed    problems (V. H. Winston & Sons, Washington, D.C.: John Wiley & Sons,    New York, 1977).-   [34] L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total    variation based noise removal algorithms,” Phys. D: Nonlinear    Phenom. 60, 259-268 (1992).-   [35] S. Mallat, A Wavelet Tour of Signal Processing, Third Edition:    The Sparse Way (Academic Press, Inc., USA, 2008), 3rd ed.-   [36] M. Elad, Sparse and Redundant Representations: From Theory to    Applications in Signal and Image Processing (Springer Publishing    Company, Incorporated, 2010).-   [37] G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G.    Dimakis, and R. Willett, “Deep learning techniques for inverse    problems in imaging,” (2020).-   [38] D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Deep image prior,”    in Proceedings of the IEEE Conference on Computer Vision and Pattern    Recognition (CVPR), (2018).-   [39] K. Monakhova, V. Tran, G. Kuo, and L. Waller, “Untrained    networks for compressive lensless photography,” Opt. Express    29,20913-20929 (2021).-   [40] S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg,    “Plug-and-play priors for model based reconstruction,” in IEEE    Global Conference on Signal and Information Processing, (2013), pp.    945-948.-   [41] Y. Romano, M. Elad, and P. Milanfar, “The little engine that    could: Regularization by denoising (red),” SIAM J. on Imaging Sci.    10 (2016).-   [42] K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep cnn    denoiser prior for image restoration,” in IEEE Conference on    Computer Vision and Pattern Recognition, (2017), pp. 3929-3938.-   [43] T. Tirer and R. Giryes, “Image restoration by iterative    denoising and backward projections,” IEEE Transactions on Image    Process. 28,1220-1234 (2019).-   [44] K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte,    “Plug-and-play image restoration with deep denoiser prior,” IEEE    Transactions on Pattern Analysis Mach. Intell. (2021).-   [45] H. Zhou, H. Feng, W. Xu, Z. Xu, Q. Li, and Y. Chen, “Deep    denoiser prior based deep analytic network for lensless image    restoration,” Opt. Express 29,27237-27253 (2021).-   [46] S. Zheng, Y. Liu, Z. Meng, M. Qiao, Z. Tong, X. Yang, S. Han,    and X. Yuan, “Deep plug-and-play priors for spectral snapshot    compressive imaging,” Photon. Res. 9, B18—B29 (2021).-   [47] H. Qiu, Y. Wang, and D. Meng, “Effective snapshot    compressive-spectral imaging via deep denoising and total variation    priors,” in Proceedings of the IEEE/CVF Conference on Computer    Vision and Pattern Recognition (CVPR), (2021), pp. 9127-9136.-   [48] X. Yuan, Y. Liu, J. Suo, F. Durand, and Q. Dai, “Plug-and-play    algorithms for video snapshot compressive imaging,” IEEE    Transactions on Pattern Analysis Mach. Intell. pp. 1-1 (2021).-   [49] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein,    “Distributed optimization and statistical learning via the    alternating direction method of multipliers,” Found. Trends Mach.    Learn. 3,1-122 (2011).-   [50] A. Beck and M. Teboulle, “A fast iterative    shrinkage-thresholding algorithm with application to wavelet-based    image deblurring,” in 2009 IEEE International Conference on    Acoustics, Speech and Signal Processing, (2009), pp. 693-696.

What is claimed is:
 1. A system for polarization imaging, comprising anoptical diffuser characterized by a point spread function (PSF), animage sensor, a spatially multiplexed polarization filter between saidoptical diffuser and said image sensor, and an image processor having acircuit configured to receive signals from said image sensor and toreconstruct based on said PSF a separate image for each polarizationdirection formed on said polarization filter.
 2. The system of claim 1,wherein said image sensor comprises a plurality of wavelength channels,and wherein said circuit of said image processor is configured toreconstruct each separate image using all wavelength channels.
 3. Thesystem of claim 2, wherein said polarization filter comprises aplurality of spatial periods, each corresponding to a different regionover said image sensor, such that a cross-section of portion of a lightbeam passing through a particular period of said polarization filtercovers a respective region over said image sensor.
 4. The system ofclaim 1, wherein said circuit of said image processor is configured forcombining said separate images to a combined image.
 5. The system ofclaim 1, wherein said optical diffuser comprises a random diffusionpattern.
 6. The system of claim 1, wherein said circuit of said imageprocessor is configured to reconstruct said image based only on signalsfrom a region over said image sensor at which said PSF isshift-invariant.
 7. The system of claim 1, wherein said circuit of saidimage processor is configured to reconstruct said image based on signalsfrom a region over said image sensor at which said PSF isshift-invariant, and a region over said image sensor at which said PSFis shift-variant.
 8. The system of claim 1, wherein said polarizationfilter is configured to apply only linear polarization.
 9. The system ofclaim 1, wherein said polarization filter is configured to applycircular and/or elliptic polarization.
 10. The system of claim 1,wherein said diffuser, said image sensor, and said polarization filterare arranged such that light arrives to said diffuser directly from ascene, then arrives to said arrived to said polarization filter directlyfrom said diffuser, then arrives to said image sensor directly from saidpolarization filter.
 11. The system of claim 1, being a lenslesspolarization imaging system.
 12. A method of polarization imaging,comprising diffusing a light beam arriving from a scene according to apoint spread function (PSF) to provide a diffused light beam, applyingspatially multiplexed polarization filtering to said diffused light toprovide a filtered diffused light beam, generating electrical signalsresponsively to said filtered diffused light beam, and processing saidelectrical signals to reconstruct, based on said PSF, a separate imagefor each polarization direction of said multiplexed polarizationfiltering.
 13. The method of claim 12, wherein said generatingelectrical signals comprises generating electrical signals over aplurality of wavelength channels, and wherein said processing comprisesreconstruct each separate image using all wavelength channels.
 14. Themethod of claim 13, wherein said applying said spatially multiplexedpolarization filtering comprises applying plurality of spatial periods,each filtering a different wavelength channel.
 15. The method of claim12, comprising combining said separate images to a combined image. 16.The method of claim 12, wherein said diffusing is according to a randomdiffusion pattern.
 17. The method of claim 12, wherein saidreconstructing is based only on signals from a portion of across-section of said beam at which said PSF is shift-invariant.
 18. Themethod of claim 12, wherein said reconstructing is based on signals froma portion of a cross-section of said beam at which said PSF isshift-invariant, and a portion of a cross-section of said beam at whichsaid PSF is shift-variant.
 19. The method of claim 12, wherein saidapplying said spatially multiplexed polarization filtering comprisesapplying only linear polarization.
 20. The method of claim 12, whereinsaid applying said spatially multiplexed polarization filteringcomprises applying circular and/or elliptic polarization.